In general, software testing is a systematic process designed to find differences between the expected behavior of a software system as specified in a software requirements document and the actual behavior of the software system. In other words, software testing is an activity for finding defects, commonly known as bugs, in the software system.
Complex software systems are tested, at least in part, using automated testing tools. More specifically, test cases are developed to test specific requirements or features of a software system. The test cases are then used as input to an automated testing tool that uses the test cases to test the software system. As the test cases are executed, the testing tool creates a test log that includes the result of executing each test case. The test log is then examined to determine if any of the test cases failed, thus indicating defects in the software system. In addition, the result of each of the failed test cases is examined to determine if the defect indicated is a known defect (i.e., a defect that was previously detected and recorded in a defect report in a defect tracking system) or a new defect.
The examination of the test log to determine if a failed test case is an indication of a known defect or a new defect is typically a manual process. That is, a software tester looks at each test result in the test log. If the test result indicates a failed test case, the software tester then determines whether the indicated defect is known by determining if there is a defect report for the defect. In some testing systems, a test case may include identifiers of any defect reports (e.g., defect report numbers) related to that test case. In such systems, the software tester can look up the test case in a test case repository and check to see if the test case includes any defect report identifiers. Further, in some defect tracking systems, an identifier for a test case (e.g., a test case number or name) related to a defect may be included in the defect report. In such systems, a software tester can search for defect reports related to a failed test case using the identifier for the test case.